Skip to content

render mcp apps inline in goose2#8877

Open
aharvard wants to merge 12 commits intomainfrom
codex/mcp-apps-step3-renderer
Open

render mcp apps inline in goose2#8877
aharvard wants to merge 12 commits intomainfrom
codex/mcp-apps-step3-renderer

Conversation

@aharvard
Copy link
Copy Markdown
Collaborator

@aharvard aharvard commented Apr 28, 2026

Category: new-feature
Tracking: #8591
User impact: Goose2 can render MCP apps inline and let users interact with them directly in the chat timeline.

Summary
This PR turns the MCP app payload that Goose2 already receives from tool results into a real inline app view. Apps are loaded through a local double-iframe sandbox proxy, initialized with Goose2 host context, and connected back to Goose through the MCP app bridge.

The result is that users can run an MCP tool, see its app UI inline, click controls inside that UI, and have those interactions route back through the current Goose session.

MCP App Bench demo

goose2-mcp-apps-inline-rendering.mov

real-world MCP App demo (agentic commerce)

goose2-mcp-apps-agentic-commerce.mov

What Reviewers Should Look For

  • Inline rendering: MCP app HTML is hydrated into McpAppView and hosted in a sandboxed iframe instead of being shown as raw data.
  • App bridge behavior: app-originated messages, nested tool calls, resource reads, link opens, and resize notifications route through Goose2 and Goose serve.
  • State preservation: live updates and replay keep the MCP app payload attached to the correct assistant message.
  • Theme and layout polish: theme changes update host context without remounting the app, transparent iframes match light/dark mode, and the inline app/composer boundary behaves cleanly.
  • Structured output: structured tool content remains visible and inspectable in the normal tool-result UI.
  • Generated/API surface: ACP schema, SDK output, and Goose2 dependency lockfile changes are included for the new bridge/proxy surface.

How Inline Apps Load

sequenceDiagram
    participant User
    participant Goose2
    participant Renderer as McpAppView / AppRenderer
    participant GooseServer as Goose server
    participant MCPServer as MCP server
    participant Proxy as Trusted proxy (/mcp-app-proxy)
    participant Guest as Guest origin (/mcp-app-guest)

    User->>Goose2: Send prompt
    Goose2->>GooseServer: Continue session with prompt
    GooseServer->>GooseServer: Decide to call MCP tool
    GooseServer->>MCPServer: Call selected tool
    MCPServer-->>GooseServer: Tool result with app resource metadata
    GooseServer->>MCPServer: Read app resource
    MCPServer-->>GooseServer: HTML, CSP, and UI metadata
    GooseServer-->>Goose2: Assistant update with MCP app payload
    Goose2->>Goose2: Normalize MCP app CSP metadata
    Goose2->>Renderer: Render inline app payload
    Renderer->>GooseServer: Request local app host info
    GooseServer-->>Renderer: Proxy base URL and secret
    Note over Renderer,GooseServer: Inline apps require a local ws/http serve origin. Secure serve URLs fail closed for now.
    Renderer->>Proxy: Load trusted outer proxy with app HTML, CSP, and secret
    Proxy->>Proxy: Verify secret and loopback peer
    Proxy->>Proxy: Store guest HTML and CSP
    Proxy->>Guest: Create sandboxed guest iframe
    Guest->>Proxy: Fetch stored guest HTML
    Proxy->>Proxy: Verify secret and loopback peer
    Proxy-->>Guest: Serve isolated app HTML
    Guest-->>Renderer: Initialize with tool data and host context
Loading

McpAppView is the Goose2 React wrapper that renders the @mcp-ui/client AppRenderer. Goose serves a trusted outer proxy at /mcp-app-proxy; that proxy stores the app document and loads it from a separate loopback guest origin at /mcp-app-guest, preserving the sandbox while keeping guest HTML off the Goose/ACP origin.

Bridge Paths Covered

  • ui/message: sends a normal user message through the existing Goose2 chat flow.
  • tools/call: calls an app-visible tool through _goose/tool/call without replacing the original app tool result.
  • resources/read: reads resources for the same session and extension through ACP.
  • ui/open-link: validates http/https links and asks the user to confirm before opening through the native shell.
  • ui/notifications/size-changed: resizes the inline frame and preserves sticky scroll behavior.
  • ui/notifications/host-context-changed: keeps the already-mounted app aligned with host theme and container changes.

Bridge Message Sequences
Any UI element in the embedded app, such as a button, can initiate these JSON-RPC bridge requests. Host notifications use the same bridge back into the already-mounted app.

ui/message

sequenceDiagram
    participant App as MCP app iframe
    participant Renderer as McpAppView AppRenderer
    participant Chat as Goose2 chat submit
    participant GooseServer as Goose server

    App->>Renderer: ui/message
    Renderer->>Chat: Submit as user message
    Chat->>GooseServer: Continue the same session
    GooseServer-->>Chat: Stream assistant updates
    Chat-->>Renderer: Update timeline
Loading

tools/call

sequenceDiagram
    participant App as MCP app iframe
    participant Renderer as McpAppView AppRenderer
    participant GooseServer as Goose server

    App->>Renderer: tools/call
    Renderer->>GooseServer: _goose/tool/call
    GooseServer-->>Renderer: Tool result
    Renderer-->>App: Resolve tool call
Loading

resources/read

sequenceDiagram
    participant App as MCP app iframe
    participant Renderer as McpAppView AppRenderer
    participant GooseServer as Goose server

    App->>Renderer: resources/read
    Renderer->>GooseServer: _goose/resource/read
    GooseServer-->>Renderer: Resource contents
    Renderer-->>App: Resolve resource read
Loading

ui/open-link

sequenceDiagram
    participant User
    participant App as MCP app iframe
    participant Renderer as McpAppView AppRenderer
    participant Modal as LinkSafetyModal
    participant Shell as Native shell

    User->>App: Click link or button
    App->>Renderer: ui/open-link with URL
    Renderer->>Renderer: Allow http and https only
    alt Allowed URL
        Renderer->>Modal: Show confirmation
        alt User confirms
            User->>Modal: Open link
            Modal-->>Renderer: Confirm
            Renderer->>Shell: Open URL
            Renderer-->>App: Return success
        else User cancels
            User->>Modal: Cancel
            Modal-->>Renderer: Cancel
            Renderer-->>App: Return error
        end
    else Blocked scheme
        Renderer-->>App: Return error
    end
Loading

ui/notifications/size-changed

sequenceDiagram
    participant App as MCP app iframe
    participant Renderer as McpAppView AppRenderer

    App->>Renderer: ui/notifications/size-changed
    Renderer->>Renderer: Update iframe height
    Renderer->>Renderer: Preserve sticky scroll when pinned
Loading

ui/notifications/host-context-changed

sequenceDiagram
    participant Renderer as McpAppView AppRenderer
    participant App as MCP app iframe

    Renderer->>Renderer: Recompute host context
    Renderer-->>App: ui/notifications/host-context-changed
Loading

Goose2 chat submit is the existing chat submission/session continuation path used by the composer; the app bridge reuses it for ui/message. LinkSafetyModal is the Goose2 confirmation UI shown before the native shell opens app-requested links.

Manual Validation
MCP AppBench server URL: https://mcp-app-bench.onrender.com/mcp

Suggested prompts after adding the server to Goose2:

  1. run the app bench messaging inspector.
  2. run the app bench host info inspector.
  3. run the app bench tool data inspector.
  4. run the app bench display modes inspector.
  5. run the app bench transparency inspector.

In the Messaging Inspector, click:

  • ui/message: a new user message should appear in Goose2.
  • tools/call: a nested tool result should return inside the app without clearing the original app footer or structured output.
  • resources/read: the demo resource should return through the session-scoped resource path.
  • ui/open-link: Goose2 should show the link-safety modal; confirming opens the URL, while canceling returns an error to the app.
  • ui/notifications/size-changed: the frame should resize and stay pinned when the user is already at the bottom.
  • notifications/message and ping: the app/server messaging path should complete successfully.

Visual checks:

  • Switch Goose2 between light and dark after an app is already loaded. The app should stay rendered, and the reported host theme should match Goose2.
  • In the Transparency Inspector, transparent areas should show the Goose2 chat background in both light and dark mode.
  • Bordered apps should get border/radius chrome. Borderless apps should not get border, radius, shadow, or extra chrome.
  • The app frame should sit naturally in the message column, and the chat input should not overlap loading labels such as Reasoning.

Not In Scope Yet
This PR establishes a working inline MCP app host, but it is not full MCP Apps conformance.

Likely follow-ups:

  • Fill out more of hostContext, especially the broader optional fields in Host Context in McpUiInitializeResult.
  • Add display mode handling beyond inline, including fullscreen and picture-in-picture.
  • Implement ui/update-model-context so apps can update model context for future turns.
  • Explore HTTPS or a stronger secured-origin model for the local proxy.

Bojun-Vvibe added a commit to Bojun-Vvibe/oss-contributions that referenced this pull request Apr 29, 2026
- BerriAI/litellm#26730: aiohttp SO_KEEPALIVE for NAT idle timeouts (merge-after-nits)
- BerriAI/litellm#26714: gate skills auto-execution and guardrail exec() behind env vars (merge-after-nits)
- google-gemini/gemini-cli#26139: FooterConfigDialog stale-closure reset bug (merge-after-nits)
- aaif-goose/goose#8877: inline MCP apps in goose2 with local axum proxy (request-changes)
@aharvard aharvard force-pushed the codex/mcp-apps-step3-renderer branch 2 times, most recently from 6500c82 to 10cf2c7 Compare April 30, 2026 13:44
@aharvard aharvard marked this pull request as ready for review April 30, 2026 14:37
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10cf2c73bd

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/acp/mcp_app_proxy.rs Outdated
Comment thread crates/goose/src/acp/server.rs Outdated
@aharvard aharvard requested a review from alexhancock April 30, 2026 15:58
@aharvard
Copy link
Copy Markdown
Collaborator Author

I got a separate critical-review pass on this PR, with the main concerns captured here: https://discord.com/channels/1287729918100246654/1499419742518251642/1499457327890174083

I pushed one focused follow-up commit that:

  • Preserves the double-iframe architecture while serving guest app HTML from a distinct loopback origin.
  • Keeps the sandboxed guest iframe isolated from ACP/tool endpoints.
  • Normalizes and validates CSP source metadata before applying it.
  • Enforces _meta.ui.visibility for app-initiated tools/call requests.
  • Makes iframe nonce handling reload-safe with TTL cleanup.
  • Removes unused Goose2 host JSON helper code.
  • Confirms Goose’s existing MCP client timeout/cancellation path covers nested app tool calls.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 977eea58c2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ui/goose2/src-tauri/src/commands/acp.rs
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
@aharvard aharvard force-pushed the codex/mcp-apps-step3-renderer branch from 977eea5 to a5e49fb Compare April 30, 2026 18:38
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

body: JSON.stringify({
secret: proxyParams.secret,
html: html
})

P1 Badge Preserve CSP when storing guest HTML in proxy template

The proxy template no longer includes a csp field in the POST /mcp-app-guest body, but store_guest_html/serve_guest_html in crates/goose-server/src/routes/mcp_app_proxy.rs rely on that value to emit a Content-Security-Policy header for the guest document. With this omission, guest HTML is served without the intended CSP restrictions, which materially weakens isolation for untrusted MCP app content.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/acp/mcp_app_proxy.rs
Comment thread crates/goose-cli/src/cli.rs Outdated
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 92ba158cb1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ui/goose2/src-tauri/src/commands/acp.rs Outdated
Comment thread crates/goose/src/acp/mcp_app_proxy.rs Outdated
Keep the MCP app renderer PR scoped to the Goose2 ACP proxy path by reverting the accidental legacy goose-server template edits.

Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62a7129707

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ui/goose2/src/features/chat/ui/useMcpAppSandbox.ts Outdated
Comment thread crates/goose/src/acp/templates/mcp_app_proxy.html Outdated
Address review feedback for the Goose2 MCP app path by preserving configured proxy path prefixes, rejecting non-loopback inline app proxy use while the guest sandbox is loopback-only, randomizing the raw goose serve fallback secret, and accepting validated CSP host-source metadata such as wildcard origins.

Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 79efe17390

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/acp/mcp_app_proxy.rs Outdated
Comment thread ui/goose2/src/features/chat/ui/McpAppView.tsx Outdated
Remove the srcdoc fallback from the ACP MCP app proxy so failed guest HTML storage cannot collapse the double-iframe isolation boundary. The proxy now renders a safe error message without executing guest HTML.

Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb452d8d5e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ui/goose2/src/features/chat/ui/mcpAppPayload.ts Outdated
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e9043d2e4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ui/goose2/src-tauri/src/commands/acp.rs Outdated
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Signed-off-by: Andrew Harvard <aharvard@squareup.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 35de9b560d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +1098 to +1099
let secret_key =
std::env::var("GOOSE_SERVER__SECRET_KEY").unwrap_or_else(|_| generate_serve_secret_key());
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject empty GOOSE_SERVER__SECRET_KEY values

Treating GOOSE_SERVER__SECRET_KEY as valid whenever the variable exists means GOOSE_SERVER__SECRET_KEY="" produces an empty shared secret, so /mcp-app-proxy and /mcp-app-guest auth can be satisfied with an empty secret parameter/body. In environments where this var is templated but left blank, any loopback client (including arbitrary web pages reaching localhost) can use the MCP app proxy endpoints without guessing a key; trim/validate and either fail startup or fall back to a generated secret when the value is empty.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant